Meta-Optimisation on a High-Performance Computing System

All optimisation algorithms have parameters that affect their performance and reliability. Usually the default values of these parameters are problem-dependent. Regardless of this fact it is common practice to use some default values that are provided with the optimisation algorithm. Finding the optimal values of these parameters is a computationally expensive optimisation problem also known as meta-optimization. The computational complexity comes from the fact that every cost-function evaluation in meta-optimisation involve several runs of an optimisation algorithm that evaluate its behavior for given values of algorithm parameters. The most common approach to making meta-optimisation feasible is the use of parallel computing. The paper presents the construction of the cost function for meta-optimisation of direct search optimisation algorithms. We demonstrate the approach by optimising the parameters of the Nelder-Mead simplex algorithm using a high-performance computing system comprising 100 processing units. The results of the meta-optimisation are surprising because the obtained values of parameters greatly differ from the values that were published 50 years ago, but are still used despite their suboptimality.